Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

internal/civisibility: add early flake detection feature #2916

Merged
merged 11 commits into from
Oct 10, 2024
Merged

internal/civisibility: add early flake detection feature #2916

merged 11 commits into from
Oct 10, 2024

Conversation

tonyredondo
Copy link
Member

@tonyredondo tonyredondo commented Oct 7, 2024 •
edited
Loading

What does this PR do?

This PR adds the Early Flake Detection feature to the CI Visibility instrumentation.

It also:

  • Fix the function endline retriever for one-liner functions.
  • Splits the instrumentation.go file and create a new instrumentation_orchestrion.go file with all the api exposed and used by the orchestrion integration.
  • Refactor the auto test retries feature to avoid code duplication merging the retry logic from both auto test retries and early flake detection.
  • Fixes minor subtests naming issues.
  • Propagate retry tags to subtests.

Motivation

It's a new core CI Visibility feature we were missing in the current implementation.

Reviewer's Checklist

  • Changed code has unit tests for its functionality at or near 100% coverage.
  • System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
  • There is a benchmark for any new code, or changes to existing code.
  • If this interacts with the agent in a new way, a system test has been added.
  • Add an appropriate team label so this PR gets put in the right place for the release notes.
  • Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.

Unsure? Have a question? Request a review!

@tonyredondo tonyredondo self-assigned this Oct 8, 2024
@tonyredondo tonyredondo force-pushed the tony/civisibility-early-flake-detection branch from 821123e to 9b80796 Compare October 9, 2024 09:32
@DataDog DataDog deleted a comment from pr-commenter bot Oct 9, 2024
@DataDog DataDog deleted a comment from pr-commenter bot Oct 9, 2024
@tonyredondo tonyredondo marked this pull request as ready for review October 9, 2024 11:33
@tonyredondo tonyredondo requested a review from a team as a code owner October 9, 2024 11:33
@tonyredondo tonyredondo changed the title internal/civisibility: add the early flake detection feature internal/civisibility: add early flake detection feature Oct 9, 2024
@pr-commenter
Copy link

pr-commenter bot commented Oct 9, 2024 •
edited
Loading

Benchmarks

Benchmark execution time: 2024-10-10 14:37:42

Comparing candidate commit 875549a in PR branch tony/civisibility-early-flake-detection with baseline commit 2f2854f in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 1 unstable metrics.

@tonyredondo tonyredondo force-pushed the tony/civisibility-early-flake-detection branch from 1077d6e to f88801b Compare October 10, 2024 13:21
@tonyredondo
internal/civisibility: add the early flake detection feature
790a602
@tonyredondo
internal/civisibility: fix endline when for one-liner functions
0661c26
@tonyredondo
internal/civisibility: add the test.early_flake.abort_reason suppor…
83d464e 
…t. Also, propagate the test is new and retry tag to the subtest spans
@tonyredondo
internal/civisibility: refactor the test suite for civisibility gotes…
6bd506e 
…ting instrumentation for multiple testing scenarios.
@tonyredondo
internal/civisibility: fix subtest unique naming on retries.
1b87a4b
@tonyredondo
internal/civisibility: ensure tag propagation to subtests and add mor…
084c340 
…e checks to the test suite.
@tonyredondo
internal/civisibility: change test
440af1d
@tonyredondo
internal/civisibility: refactor auto flaky test retries and early fla…
348add1 
…ke detection code to avoid code duplication.
@tonyredondo
internal/civisibility: split instrumentation.go in two files with ins…
da42956 
…trumentation_orchestrion.go representing the api exposed for the orchestrion integration.
@tonyredondo
internal/civisibility: test fixes
78a7f85
@tonyredondo
internal/civisibility: changes from review
875549a
@tonyredondo tonyredondo force-pushed the tony/civisibility-early-flake-detection branch from f88801b to 875549a Compare October 10, 2024 13:53
@tonyredondo tonyredondo merged commit 5b9a8af into main Oct 10, 2024
144 of 145 checks passed
@tonyredondo tonyredondo deleted the tony/civisibility-early-flake-detection branch October 10, 2024 15:19
MNThomson pushed a commit to agilebits/dd-trace-go that referenced this pull request Oct 21, 2024
@tonyredondo @MNThomson
internal/civisibility: add early flake detection feature (DataDog#2916)
0164d23
darccio pushed a commit that referenced this pull request Oct 23, 2024
@tonyredondo @darccio
internal/civisibility: add early flake detection feature (#2916)
35b999e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vitor-de-araujo vitor-de-araujo vitor-de-araujo approved these changes

@nikita-tkachenko-datadog nikita-tkachenko-datadog Awaiting requested review from nikita-tkachenko-datadog nikita-tkachenko-datadog is a code owner automatically assigned from DataDog/ci-app-libraries

Assignees

@tonyredondo tonyredondo

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@tonyredondo @vitor-de-araujo